Repository navigation
Conversation
Move the transfer logic of lfsd into a separate Go module, unknwon.dev/git-lfs-server/lfs, so that other Git servers can serve LFS with the same storage backends. The module has an lfs.Service that answers batch, upload, download, and verify requests on top of an lfs.Store for metadata and a storage.Backend for content, and leaves routing, authentication, and response encoding to the caller. lfsd now builds one lfs.Service per forge host and keeps its HTTP responses, except that a proxied upload of an already stored object must send matching content before the object is linked to another repository. Previously, any body was accepted in that case, so a writer of one repository could gain read access to any object of another repository by its OID and size.
Windows cannot remove the test's temp dir while the object file is open.
Add Service.Serve, which handles the batch, upload, download, and verify requests of a repository given an Access that the caller resolved after authenticating the request. It dispatches on the method and the last path segment, so it works with any router that runs net/http handlers and any URL layout. lfsd now only maps its forge permissions to an Access and hands requests to it. Upload actions now set Content-Type to application/octet-stream so the Git LFS client does not send a type detected from the file content.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Moves the LFS logic of lfsd, including its HTTP handling, into a separate Go module,
unknwon.dev/git-lfs-server/lfs, so Gogs (and other Git servers) can serve Git LFS with the same storage backends, including S3-compatible presigned storage.lfs.Service.Servehandles the batch, upload, download, and verify requests of a repository. The caller routes and authenticates the request, then passes anlfs.Access(repository key, base URL of its endpoints, read and write permission). It dispatches on the method and the last path segment, so it works with any router that runsnet/httphandlers (Flamego, Macaron,http.ServeMux) and any URL layout.Servesits onBatch,Upload,Download, andVerify, which stay exported for servers that need their own HTTP handling. They work on anlfs.Store(metadata) and astorage.Backend(content).lfs/storage(moved frominternal/storage) andlfs/internal/iox(moved frominternal/iox) ship with the module.URI(oid)moved fromPresignerup toBackend.Service.PreviousStorageskeeps objects readable after the backend for new uploads changes, e.g., local disk to S3.Service.CrossRepoDedupis on in lfsd (current behavior). Servers whose metadata cannot share objects across repositories leave it off.go 1.26, so importers do not pull in PostgreSQL, Flamego, or the forge clients, and do not need the Go version of lfsd. CI, Dockerfile, and moon cover both modules.lfsd builds one
lfs.Serviceper forge host, maps forge permissions tolfs.Access, andinternal/database.DBimplementslfs.Store.Behavior changes in lfsd
FilesystemBackend.Putreturned early without reading the body, so any writer of one repository could link any known OID and size to their repository with arbitrary bytes and then download the original content. Gogs fixed the same issue in its own LFS code. Presigned uploads were not affected.Content-Type: application/octet-stream, so the Git LFS client does not send a type detected from the file content. Gogs requires it on uploads, and it is not part of the presigned signature.[lfs]instead of[batch],[upload],[download], and[verify].Test plan
moon run :lint :testfor both projects, andgo test -raceinlfs.lfs/service_test.gocovers proxy and presign flows against the real filesystem and gofakes3 backends. Mutation checks: removing the body drain, or forcing cross-repo dedup on, fails the tests.lfs/http_test.gocoversServe: dispatch, permission checks, status codes, the error format, credential forwarding into actions, and redirects for presigned storage.mainand on this branch, on PostgreSQL 17 with filesystem and gofakes3 storage. Status codes, headers, and bodies match except the two changes above.git-lfs3.6.1 push and clone through lfsd for filesystem and S3 presign.Servethrough a localreplace(Gogs changes not in this PR). Objects written by the old Gogs still download, local and S3 round trips pass, and with S3 pushes go directly to the bucket and downloads redirect with 307. Gogs moves to the spec response format: its per-object errors were underactions.error, which the Git LFS client never reads, so a missing object used to leave the pointer file with exit 0 and now fails withobject does not exist.